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ABSTRACT 

Technology and change are so closely related that the use of the word innovation seems synonymous with 
technology in many contexts, including that of higher education. This paper contends that university 
culture and existing capability constrain such innovation and to a large extent determine the nature and 
extent of organizational change. In the absence of strong leadership, technologies are simply used as 
vehicles to enable changes that are already intended or which reinforce the current identity. These 
contentions are supported by evidence from e-learning benchmarking activities carried out over the past 
five years in universities in the United States, the United Kingdom, Australia and New Zealand. 
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I. INTRODUCTION 

Educational technology is a field based on change. [1, p. 933] 
Investment in technology systems by universities is driven by the expectation that increased use of 
technology will improve the quality and flexibility of learning [2-7]. This investment has been supported 
by the widespread adoption of Learning Management Systems and the computerization of key 
administrative functions [8, 9], and the maintenance of an effective technology infrastructure remains a 
key strategic focus for university leaders [10, 11]. 

However, while information technology systems have become mainstream, and even strategically 
irrelevant for many activities [12, 13], changes in the experience of learning and teaching enabled by 
technology are less apparent. As educational technologists we share a common belief that technology can 
significantly improve the experience of learners and teachers. That belief has seen the field of educational 
technology progress from an initial naivety that simply introducing technology would improve the 
learning experience. Many institutions have attempted variations on early adopter innovation projects, 
invested in substantial infrastructures, and consequently developed an awareness of the need for 
formalized and systematic professional development. Beyond the focus on the individual teacher we also 
have the rhetoric of organizational change, perhaps best expressed by Fullan: 

The answer to large-scale reform is not to try to emulate the characteristics of the minor-ity who 
are getting somewhere under present conditions ... Rather, we must change existing conditions so 
that it is normal and possible for a majority of people to move forward. [14, p.268] 

This awareness of the role of organizational change is not recent, with the need to go beyond the 
pedagogy and technology to consider organizational aspects [2, 7], leadership [15, 16] and systems [17- 
20] recognized. Those leading change in universities need to balance technological, academic and 
administrative concerns [11, 21]. These include the strategy and culture of the organization [18, 16, 22], 
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resource utilization [23], sustainability [24, 25], scalability and reusability [26, 27]. 

This paper explores evidence from e-learning benchmarking activities carried out over the past five years 
in universities in the United States, the United Kingdom, Australia and New Zealand in an attempt to 
understand the nature of university change in response to technology. 

II. ORGANIZATIONAL CHANGE AS A THEORETICAL 

CONSTRUCT 

Simple understandings lead to general rules to be applied in all situations; 
complicated understandings suggest that situations differ and that reliance 
on experiences of the past may prove dysfunctional. [...] Only complicated 
understandings can see the many and conflicting realities of complicated 

situations. [28, p. 209] 

Change in large complex organizations can be described as operating at multiple levels: process, systems, 
structures, organizations and institutions [29, 30]. Process and systems change happens frequently as 
new ideas, technologies or capabilities become available within an organization and can be driven by 
individuals or small groups within the organization. When these changes are not disruptive to the work of 
other systems and structures, they can be described as interoperable change. Historically, structural and 
organizational change has been managed as a discontinuous process with organizations undergoing a 
period of instability and change before returning to periods without change. 

Birnbaum [28] suggests that change can occur more effectively if universities are managed according to 
the principles of cybernetics, what Stafford Beer called the science of effective organization. Structures 
within the organization are then organized in a loosely coupled manner [31]. Christensen, Anthony, and 
Roth [32] identify three types of organizational change. ‘Sustaining’ changes are ones that improve the 
function of the organization in ways that are consistent with previous activities. Putting lecture handouts 
on a Learning Management System rather than expecting students to pick up a physical copy from an 
office is an example of a sustaining change. ‘Disruptive’ changes create new markets or reshape existing 
ones. ‘Low-end disruptive’ changes can occur when the existing product exceeds the needs of some 
consumers. For example, many people obtaining degrees might be well served by less comprehensive 
providers with fewer subject options but also significantly lower fees. ‘New market disruptive’ change 
can occur when consumers are limited by the capability or complexity of existing products. Online 
delivery of degrees is potentially a new market disruption, although the lack of success of virtual 
universities to date illustrates the challenge facing providers attempting to realize that potential [33, 

9]. 

Disruptive change is problematic for dominant organizations as the natural tendency is to protect existing 
structures and activities, particularly when those are currently seen as successful. It is important to 
emphasize that technology is by default neither sustaining nor disruptive in nature. The use of the Internet 
to deliver courses can be a low-end disruptive change to universities when implemented by an 
organization to undercut the costs of obtaining a basic degree; or simply a sustaining change when used 
by an existing distance university to supply electronic rather than paper documents. Christensen, 
Anthony, and Roth [32, p.126] contend that the use of the Internet to deliver university education has 
failed in many cases as it has been used to deliver an inferior experience in competition to the existing 
model to the same group of students, rather than to current ‘non-consumers’. They suggest that such 
failures, rather than being the result of poor course design, are the result of a misidentification of the 
market for such courses. 


III. MUST UNIVERSITIES CHANGE AND SHOULD CHANGE BE A 

RESPONSE TO TECHNOLOGY? 


An obvious question to ask is whether universities need to and should change in response to external 
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forces, including technology. As an institution of society [30], the university has existed for centuries. A 
university degree is considered an essential qualification for many young people, and accredited degrees 
offer many international opportunities for those who wish to travel and work. The stability of the 
institution and the apparent resistance to change may reflect the value to society of the institution in its 
current form. 

One obstacle to change may be the lack of strong evidence that technology is benefiting educational 
outcomes for students [34-39, 9 ]. While this is most probably a result of poor research design and 
the challenge of collecting empirical evidence of improvement in student learning outcomes, it does make 
it harder to justify the expense and disruption of change to skeptical organization leaders, colleagues and 
other stakeholders. 

There is also the need to consider the purpose of change. Much of the recent change evident in 
universities has been driven by financial accountability for public funds. The university has a range of 
purposes, participants and audiences, all of whom apply pressures for change to better suit their needs or 
resist changes that disrupt their perception of the university as an institution [40, 41]. Perkins [42, p.12] 
recognized that these conflicting purposes challenge universities, preventing the use of simple measures 
of performance such as profit, while there is also the internal conflict between structures that most 
efficiently support research with those that integrate research with education. 

Despite these internal complexities, the forces driving change are such that universities cannot ignore the 
implications. Governments are acutely conscious of the need to educate adults failed by existing 
institutions [43], and internationally there are many countries seeking solutions to the problem of 
educating a population without the resources or opportunities of university education [44]. Christensen’s 
model predicts that the risk for universities lies in the emergence of new institutions and organizations 
that meet the needs of these non-participants [32]. It is a short step from serving this group to more 
cheaply and efficiently serving significant proportions of the university’s current student population. 

Beyond the social drivers for change, there is also the rapid pace of technological change. Our society is 
likely to experience the same level of technological change in the next 100 years as it experienced in the 
past 20,000 years [45, p.50]. 

Taylor [46] observed that the challenge facing universities trying to best use technology for education is 
not so much about the innovation itself. Key is the execution of the change; the need for the organization 
to rapidly evolve to sustain the execution of change at the same increasing pace at which new 
technologies are developed. 

The ubiquity of technology, combined with ever decreasing costs, means that the use of the technology 
itself is increasingly not a distinguishing factor for organizations, but rather the quality of that use and the 
way it advances the goals of a change-resilient organization become essential to success [12, 47]. 
Organizational leaders need to distinguish the products of change, the visible uses of technology, from 
the processes that result in their use. Demers [48, p.34] identifies from the work of DiMaggio and 
Powell [49] three mechanisms that drive institutional change and which can influence processes for 
change within organizations: 

• coercive isomorphism, which explains change as a consequence of formal or informal pressures 
imposed on organizations externally; 

• mimetic isomorphism, which describes the tendency for organizations to adopt ‘standard’ or 
popular approaches in uncertain times; and 

• normative isomorphism, which is change that is driven by professionalism and the emergence of 
‘legitimated professional practices’ that result in pressure for organizations to conform because 
their staff are able to draw on organized professional networks and professional standards that 
guide their activities. 

The last of these, ‘normative isomorphism’, is the form most consistent with the values of the collegial 
university institution. ‘Coercive isomorphism’ is the form that is most visible through the pressures 
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placed by governments on (particularly public) universities to demonstrate economic benefits of their 
programs and to minimize costs. ‘Mimetic isomorphism’ is seen in the restructuring work of consultants 
[50] and the adoption by universities of popular management fads. 

Coercive isomorphism and mimetic isomorphism both represent significant challenges to the identity of 
universities as collegial, academic organizations. Resisting these forms of change behavior requires 
strong leadership, an organization with a strongly expressed collective identity, or both. 

Change within an organization can be described as top-down (driven by management), bottom-up 
(reflecting emergent or participatory-driven change), or combinations of the two. Bottom-up initiatives 
are generally driven by individual ‘early adopters’ [51], and while substantial resources have been 
invested in such projects, wider adoption and use requires more than resources: leadership, systems 
and a supportive climate for change are essential [52]. Strong leadership, aware of the dynamics and 
culture of their organization, is vital, enabling creation of systems for change and providing the resources 
needed to enact new ideas, but even with leadership change can be challenging for universities. Birnbaum 
[28, p.205] noted that leaders can drive significant change from the top most easily in universities that: 

• are in a state of acknowledged crisis; 

• are small; 

• are conspicuously out of date; or 

• have autocratic leadership. 

Other than small institutions, these conditions cannot be seen as particularly desirable or consistent with 
the collegiality of the university institution. Clearly, universities need change strategies operating 
simultaneously and synergistically at multiple levels [52-54]. Maturity models such as the e-leaming 
Maturity Model described below attempt to make explicit these strategies and capture the extent to which 
they are evident in the activities of individual organizations. 

IV. ANALYZING THE CAPABILITY OF UNIVERSITIES TO CHANGE 
WITH THE E-LEARNING MATURITY MODEL 

The e-leaming maturity model (eMM) [55] (http://www.utdc.vuw.ac.nz/research/emm/) is a quality 
improvement framework based on the ideas of the Capability Maturity Model [56] and the Software 
Process Improvement and Capability Determination (SPICE) framework {57, 58]. The eMM framework 
measures the capability of institutions to sustainably engage in technology-supported learning and 
teaching, and visualizes that capability in a way that assists leaders and managers using that information 
to undertake systematic and targeted improvements in their organization’s e-learning activities. The eMM 
is intended to go beyond benchmarking to actively support organizational change activities. The maturity 
model theoretical constmct is explicitly designed to encourage self-reflection and the improvement of 
organizational activities. 

Since the initial development, the eMM has been developed, refined and validated through a series of 
projects conducted in New Zealand [59, 60, 61], Australia [61], the United Kingdom [63. 64; 65] and 
the United States [66]. The eMM assesses the ability of an organization to ensure that e-learning design, 
development and deployment is meeting the needs of the students, staff and the organization itself in five 
dimensions (Figure 1). Rather than levels, which imply a hierarchical model of process improvement 
where capability is assessed and built in a layered and progressive manner, the concept underlying the 
eMM’s use of dimensions is holistic capability. Each process is assessed from the synergistic perspectives 
of Delivery, Planning, Definition, Management and Optimization. 
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Figure 1. eMM process dimensions. 


The eMM divides the capability of institutions to sustain and deliver e-learning into 35 processes grouped 
into five major categories or process areas that indicate a shared concern. It should be noted, however, 
that all of the processes are interrelated to some degree, particularly through shared practices and the 
perspectives of the five dimensions. Each process in the eMM is broken down within each dimension into 
practices that define how the process outcomes might be achieved by institutions (Figure 2). The practice 
statements attempt to capture directly measurable activities for each process and dimension. The 
processes and practices are derived from an extensive review of the literature, international workshops 
and experience from their application [67, 68]. 


Process 

Area 


belongs 

to 


Process 


assessed 

by 


Dimension 



defined for 
process by 


Practice 

Statements 


Figure 2. Relationships between processes, practices and dimensions. 

The eMM promotes organizational change in this framework at three main levels. The most explicit is the 
Optimization dimension, which is explicitly assessing systematic change in the organization. Beyond this, 
experienced practitioners undertake assessments using an adequacy scale that can easily evolve to reflect 
better knowledge about effective technology use. Finally, the processes and practices of the model are 
evolving, changing in subsequent versions to reflect the experience and evidence from international 
assessments and the literature [67, 68]. 

Figure 3 shows a summary of the capabilities of a number of international universities. The organizations 
shown range from very large to very small, from strongly teaching focused through to prestigious 
research-dominated universities. All of these universities describe themselves as modern organizations 
keen to take best advantage of technology to advance all of their activities. As a sample, they illustrate 
the range of organizational capability for the adoption and use of technology within the international 
university sector [69]. The capability assessments shown below focus on e-learning activities and do not 
directly assess other types of learning and teaching. 
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Figure 3. eMM capabilities of international universities [69] 
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The patterns of stronger capability (dark shading) and weaker (light shading) can be observed from this 
matrix. Each box represents a further summary of detailed practices for each dimension of the individual 
processes (rows) that can be used by institutions as checklists or plans for improvement. Each column 

represents a single institution in summary, with five sub-columns for each of the five dimensions. 

Stronger capability can arise for a number of reasons. It may reflect existing systems essential to all 
forms of delivery. For example, process 08 (Figure 4) is relatively strong in all of these institutions, as a 
result of common administrative processes being used by these universities regardless of the mode of 
delivery. The strength apparent in process S2 (Figure 4) is similarly a result of the move to support 
technology use by all library users, not just those engaging in technology-mediated learning. In both of 
these cases, however, the lack of capability in the Optimization dimension is evident, due to the absence 
of activities aimed at systematically improving the experience of technology-mediated learners. 

Stronger capability may also be a result of activities being under the direct control of individual staff able 
to influence the experience of the students through their own work. This is apparent in the generally 
stronger capability of the Delivery dimension of the Learning process area and arises from the actions of 
individual teachers within their courses and programs. The absence of capability in most institutions 
through the other dimensions shows the lack of institutional engagement with these same issues. 

Finally, stronger capability may also reflect dedicated investment of resources aimed at the process 
activities. This can be seen in the relative strength of process D5 (Figure 5), which arises from the 
consistent and substantial investment in Learning Management Systems and the supporting infrastructure 
seen in most universities. This contrasts with process D6 (Figure 5), where the comparatively weaker 
capability results from the lack of formal engagement with standards and a dependence on a specific 

vendor’s products as default solutions. 
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Figure 4. University process 08 and S2 capabilities 


Weaker capability can reflect the reality that specific processes are not useful as currently formulated (i.e. 
process D7), but can also illustrate the absence of particular activities relevant to organizational change. 
The very weak capability of the Evaluation processes and the Management dimension of all processes 
result from the widespread lack of consultation with staff and students regarding their experiences of 
technology-mediated learning. Most institutions appear to operate on the implicit assumption that their 
systems and tools are sufficient, and if there is an issue someone will complain. 
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D5: All elements of the physical e-leaming 
infrastructure are reliable, robust and sufficient 
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D6: All elements of the physical e-learning 
infrastructure are integrated using defined 
standards 
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Figure 5. University process D5 and D6 capabilities [69] 

The assessment data shown here can also be used to reflect on the types of organizational change these 
universities have demonstrated, and to suggest where opportunities and challenges await in the future. 
Figure 3 potentially represents a form of organizational change snapshot. The capability assessments 
summarizing the impact of change that has occurred to date as these universities have worked to integrate 
technology into their activities, overlaid on the generic capabilities intrinsic to each organization. 

The absence of a formal change mechanism promoting technology for learning and teaching is 
evident in the lack of capability in the Definition dimension for most of these universities. This is a 
result of there being few formal processes, policies or staff development opportunities addressing 
the implications of technology use by these organizations. Individual institutions show different patterns 
of engagement with technology suggesting the change approach being used (either explicitly or 
implicitly). University NZ-A has a distinctly stronger capability in the Delivery and Planning dimensions 
of the Support process area, consistent with an approach driven by a focus on student support, 
independent of specific courses and programs. In contrast, university NZ-B shows stronger capability in 
the Learning and Development areas, reflecting a formal design and development approach driven by a 
specialist team. The absence of capability in the Definition dimension for NZ-B, however, 
illustrates a common problem with the use of dedicated technologists, as there is no evidence of a 
mechanism transferring expertise to all teaching staff, limiting the impact by the size of the team of 
support staff. 

There is very little evidence of a culture of critical self-reflection within the organizational assessments, 
and this is shown in the almost complete absence of capability in the Optimization dimension of most 
institutions. This is possibly linked to the weak Evaluation process and Management dimension capability 
noted above, but also probably reflects the lack of attention by university leaders to these areas. This lack 
of leadership is also evident in the weak capabilities for processes 09 and 02 (Figure 6) in most of the 
institutions. This lack of strategic context is possibly one of principle reasons that technology has not 
driven organizational change, despite the opportunities to do so [11]. 
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D5: All elements of the physical e-leaming 
infrastructure are reliable, robust and sufficient 
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09: E-leaming initiatives are guided by 
institutional strategies and operational plans 
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Figure 6. University process 02 and 09 capabilities [69] 
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VI. CONCLUSIONS 

Future technological developments will probably change the nature of the student experience and 
possibly disrupt the university as an institution of society, making existing organizational models and 
pedagogies unsustainable [70]. This paper contends that, more commonly and for the immediate future, 
the change resulting from new technologies depends on the change culture and leadership decisions of the 
individual universities. 

The eMM is designed as a tool for supporting change informed by the benchmarking analysis. The 
analysis of the visualizations presented above provides the first step of a potential change process. The 
audience for this analysis encompasses all of the stakeholders, and the visual approach adopted is 
intended to make the initial questioning of organizational capability a simple pattern-matching activity. 
Once a pattern has been identified, the eMM provides the details of the underlying practices that both 
provide a framework for assessment, but also a starting point for the development of action plans [67]. 
The practices are defined so as to enable organizations considerable flexibility in how he assessed 
outcome is achieved, supporting a range of pedagogies, technologies and other significant characteristics 
of the learning and teaching environment. 

The eMM does not, however, provide a specific, pre-defined, mechanism for organizational change. The 
intention is to ensure that change is well-informed, facilitated by the ability to see areas of organizational 
strength and weakness, and that the impact of change initiatives be apparent in subsequent assessments. 
The underlying model of organizational maturity is descriptive of the characteristics of more effective 
institutions, but does not define or measure the path taken to achieve that outcome. 

The evidence from the eMM assessments presented in this paper suggest that, to date, decisions have 
been made primarily with the intention of engaging in ‘sustaining’ and ‘mimetic’ forms of change and 
that universities as organizations as yet show little capability to disrupt their existing educational models. 
An ongoing priority for the eMM research is testing this analysis, working with organizations changing to 
take advantage of new technologies, examining the ways which that work is seen in eMM capability 
assessments and verifying the utility of the eMM framework as a change-oriented quality improvement 
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tool. 

Laurillard [18] identified over a decade ago the need to examine the limitations of current practices 
carefully so as to incrementally improve capability, but the lack of engagement with students and staff 
seen in these assessments and the weak assessments in the Optimization dimension offer little evidence of 
critical self-examination. The UK Government, the Australian Government and the New Zealand 
Government have repeatedly imposed various performance indicator-reporting requirements on 
universities, but there is no evidence of a culture of systematic self-improvement driving the examination 
and use of technologies for learning and teaching. Most of the universities assessed here have supported 
staff engaging in bottom-up, early adopter innovation, but have failed to provide systems and 
environments that result in wider adoption of successful ideas [53, 54, 52]. 

The implications for those interested in promoting change within the university as an institution are 
challenging. A clear avenue for further action is finding ways that the experiences of students and staff 
can be used to frame future technology supported organizational and pedagogical change. Beyond this 
there is a need for clearly articulated goals for change supported by, rather than led or in response to 
technology, or coerced by external drivers. Reasons for change enabled by and making effective use of 
technologies are needed that are relevant to the organizations and to the staff and students who, 
collectively with the leadership, create the university [11] 

As a societal institution, universities need to grapple with the two types of disruptive change identified by 
Christensen, Anthony, and Roth [32]. University leaders need to be alert to the ways by which other 
organizations are meeting the educational needs of people who are not university students, and avoid the 
myopia common to dominant organizations [71]. Universities may not be responsible for meeting the 
needs of all adult ‘non-consumers’ of higher education, but all stakeholders in the university institution 
need to be aware that others meeting those needs might also better meet the needs of many existing 
university students. New Tow-end’ providers may lack prestigious reputations, and may well compromise 
some aspects of quality university learning, but only a very few universities can depend on their reputation 
and not respond to the demands of governments and students to deliver a more flexible and less expensive 
education. 
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